Sparsity Increases Uncertainty Estimation in Deep Ensemble

نویسندگان

چکیده

Deep neural networks have achieved almost human-level results in various tasks and become popular the broad artificial intelligence domains. Uncertainty estimation is an on-demand task caused by black-box point behavior of deep learning. The ensemble provides increased accuracy estimated uncertainty; however, linearly increasing size makes unfeasible for memory-intensive tasks. To address this problem, we used model pruning quantization with a analyzed effect context uncertainty metrics. We empirically showed that members’ disagreement increases pruning, making models sparser zeroing irrelevant parameters. Increased im-plies uncertainty, which helps more robust predictions. Accordingly, energy-efficient compressed appropriate uncertainty-aware

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Sparsity-driven weighted ensemble classifier

In this study, a novel weighted ensemble classifier that improves classification accuracy and minimizes the number of classifiers is proposed. Proposed method uses sparsity techniques therefore it is named sparsity-driven weighted ensemble classifier (SDWEC). In SDWEC, ensemble weight finding problem is modeled as a cost function with following terms: (a) a data fidelity term aiming to decrease...

متن کامل

Maximal Sparsity with Deep Networks?

The iterations of many sparse estimation algorithms are comprised of a fixed linear filter cascaded with a thresholding nonlinearity, which collectively resemble a typical neural network layer. Consequently, a lengthy sequence of algorithm iterations can be viewed as a deep network with shared, hand-crafted layer weights. It is therefore quite natural to examine the degree to which a learned ne...

متن کامل

Learning Structured Sparsity in Deep Neural Networks

High demand for computation resources severely hinders deployment of large-scale Deep Neural Networks (DNN) in resource constrained devices. In this work, we propose a Structured Sparsity Learning (SSL) method to regularize the structures (i.e., filters, channels, filter shapes, and layer depth) of DNNs. SSL can: (1) learn a compact structure from a bigger DNN to reduce computation cost; (2) ob...

متن کامل

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks. Quantifying predictive uncertainty in NNs is a challenging and yet unsolved problem. Bayesian NNs, which learn a distribution over weights, are currently the state-of-the-art for estimating predictive uncertainty; however these require significant modifica...

متن کامل

Uncertainty in Deep Learning

Deep learning has attracted tremendous attention from researchers in various fields of information engineering such as AI, computer vision, and language processing [Kalchbrenner and Blunsom, 2013; Krizhevsky et al., 2012; Mnih et al., 2013], but also from more traditional sciences such as physics, biology, and manufacturing [Anjos et al., 2015; Baldi et al., 2014; Bergmann et al., 2014]. Neural...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Computers

سال: 2021

ISSN: ['2073-431X']

DOI: https://doi.org/10.3390/computers10040054